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Abstract 


Like the Polak-Ribiére-Polyak (PRP) and Hestenes-Stiefel (HS) meth- 
ods, the classical Liu-Storey (LS) conjugate gradient scheme is widely be- 
lieved to perform well numerically. This is attributed to the in-built capa- 
bility of the method to conduct a restart when a bad direction is encoun- 
tered. However, the scheme’s inability to generate descent search direc- 
tions, which is vital for global convergence, represents its major shortfall. 
In this article, we present an LS-type scheme for solving system of mono- 
tone nonlinear equations with convex constraints. The scheme is based on 
the approach by Wang et al. (2020) and the projection scheme by Solodov 
and Svaiter (1998). The new scheme satisfies the important condition for 
global convergence and is suitable for non-smooth nonlinear problems. Fur- 
thermore, we demonstrate the method’s application in restoring blurry im- 
ages in compressed sensing. The scheme’s global convergence is established 
under mild assumptions and preliminary numerical results show that the 
proposed method is promising and performs better than two recent meth- 
ods in the literature. 
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1 Introduction 


In this paper, the following constrained system of nonlinear equations is con- 
sidered: 
F(z)=0, «we, (1) 


where F': R® > R® is a nonlinear mapping, which is continuous and mono- 
tone, namely it satisfies the inequality 


(F(z) — F(y))" (@—-y) 20, Va,y ¢ R®. (2) 


Also, ® in (1) is nonempty, closed convex set, which is sometimes expressed 
as 
®={tER”:u<a< rv}, (3) 


where u and v stand for the lower and upper bounds on the vector x. More- 
over, when ® is expressed as (3), the problem in (1) is refereed to as box 
constrained nonlinear system. 

Several applications in science, engineering and other areas of human en- 
deavour involve the system of equations represented in (1). For example, in 
problem of radiactive transfer and transport theory [15], the popular Chan- 
rasekhar integral equations is discretized and expressed as (1). Also, the 
economic equilibrium problems studied in [5, 24], are reformulated as prob- 
lem (1). In addition, some ¢; — norm regularized optimization problems in 
signal and image processing [23, 46] are obtained by reformulating systems 
of monotone nonlinear equations. 

In order to solve (1), various iterative schemes have been proposed over 
the years. Newton-type schemes [32] and their improved variants, the quasi- 
Newton schemes [4, 17] are the most widely used due to their rapid con- 
vergence properties. These methods, however, are not suitable for large- 
dimension problems due to their huge matrix storage requirement. The con- 
jugate gradient (CG) method, by virtue of its low memory requirement is the 
proper choice for problems with large dimensions. The scheme was primarily 
developed to solve the unconstrained optimization problem 


mip f@) a 


where f : R” —> R denotes a nonlinear mapping that is assumed to be 
at least twice continuously differentiable and bounded below. The following 
notations are used with respect to problem (4): 


on = ove) = Vi (ee), Ge-1 = G(@a-1); Ye—-1 = De — Gat 
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The scheme is implemented using the iterative formula 
ro ER", they =@et+ Sze, Se= Onde, k=O0,1,..., (5) 


where x; denotes the k*” iterate, a, represents a step-size that is computed 
using an appropriate line search technique, and dx is the CG search direction 
defined by 

dy =—gr + Brde-1, do =—g9o, (6) 


and {, is a scalar known as the CG (update) parameter, which defines each 
CG scheme and influences its performance. As part of the conditions for 
the sequence of iterative points {x,} generated via (5) and (6) to converge 
globally, the following sufficient descent condition is required: 


di 9k S —T\ge\|?, 7 > 0. (7) 


Due to the fine attributes of CG methods with the known fact that the first 
order optimality condition for (4), namely, V f(a) = 0, is equivalent to (1) 
with F = Vf denoting the gradient of some nonlinear functions, different 
adaptations of the scheme for solving (1) have been proposed over the years. 
Search directions of these adaptations are defined as 


—F, if k=O; 
ne res (8) 
—Fy,+ Bydp_1, otherwise, 


where Fy, = F(xx). 


One of the CG adaptations that has gained attention of researchers in 
the last decade is the three-term methods for solving (1). Wang et al. [35], 
proposed a self-adaptive three-term nonlinear CG method for solving convex 
constrained system of nonlinear equations. Search direction of the scheme 
was obtained by employing an adaptive technique and under mild conditions, 
its global convergence was established. Based on the hyperplane projection 
scheme [32], Gao and He [9], proposed a three-term modified CG method 
for solving (1). Due to its derivative-free and low storage requirement, the 
method is also ideal for nonsmooth nonlinear problems. Motivated by the 
works in [1, 13, 27, 47], Koorapetse and Kaelo [25] also proposed three-term 
adaptation of CG projection methods for solving (1). The proposed methods 
were proven to satisfy the global convergence condition and under suitable 
assumptions, the authors showed that the schemes converge globally. For 
more details see ([44, 37, 38, 39, 40, 31, 41, 11, 42, 10, 43, 12, 33, 26] ). 


In this paper, our aim is to develop an efficient three-term adaptation of 
the Liu-Storey (LS) [22] CG method for solving (1). Our inspiration comes 
from the work of Wang et al. [36] and the projection method by Solodov 
and Svaiter [32]. Apart from developing a new scheme that satisfies the 
condition for global convergence, a notable contribution of this research is 
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its application in restoring blurry images, which is a trend in compressed 
sensing. 

The paper is organized as follows: Some preliminaries leading to deriva- 
tion of the proposed method are given in Section 2. The proposed method 
and its convergence analysis are presented in Section 3. Numerical results 
and their discussions are presented in Section 4, while application of the pro- 
posed scheme is discussed in Section 5. Concluding remarks are made in 
Section 6. 


2 Preliminaries 


In the remaining part of the article, ||z|| = V2? x, stands for the £2 norm, 
Py_1 = F(ag_-1), and s,-1 = _ — Xp_-1. The following assumptions will be 
required later in the article: 


(i) The solution set ® is not empty, namely, there exists  € ® such that 
F(z) =0. 


(ii) The mapping F' satisfies the Lipschitz continuity property; i.e, there 
exists a positive constant L such that for all 2,y € R”, the following is 
satisfied: 

F(z) — F(y)|| < Lie — yl). (9) 


We now introduce the projection operator. Let ® C R” be a nonempty, 
closed and convex set. Then for each vector x € R”, its projection onto © is 
given by 

Pa(x) = arg min||x — y||: y € ®. 
Ps :R” > is referred to as the projection operator, with the nonexpansive 
property given by, 


||Pa(z) — Pe(y)|| <ll@-yll, Va,yeR", 
for which, we can write 


|Po(z)—yl| <|la—yll, Vy e ®. (10) 


3 Motivation, algorithm and global convergence of the 
new method 


This section deals with motivation of the scheme, its algorithm and conver- 
gence analysis. First, we introduce the spectral CG method, which is an 
extension of the classical scheme for solving (4). By employing the spectral 
gradient scheme by Barzilai and Bowein [2], Birgin and Martinez [3] devel- 
oped a spectral CG (SCG) method with the following search direction: 
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dy = —O9n9n + Prdk—1, 


where the update parameter /; is defined by 
(PxYe—1 — Sk—1)~ 
di 1Yk-1 


Gk 


7 


Be = 


and 6; is the spectral parameter given as 


7 
$K—15k—1 


O, = 
ie : 
Sp—1Yk-1 


The scheme is computationally efficient, but its search directions are generally 
not descent directions, i.e, the scheme does not satisfy the inequality defined 
in (7). Over the years, three-term SCG schemes that satisfy the condition (7) 
have been developed. By modifying the classical PRP CG method [29, 30], 
Li et al. [21] proposed a spectral three-term scheme with the following search 
direction: 


dy ad RIK + BMPRP d._1 —veyr-1 k > 1; 
* "l-g6, if k=0, 


7 T 
MPRP Ie YR-1 Gj, U1 


= » US » > 0. 
glog de—1| + llgx—all? ylon de—1| + Ilgx—all? 


Simple inspection reveals that the method satisfies the sufficient descent con- 
dition (7). Also, by employing standard assumptions, the authors proved 
global convergence of the scheme for uniformly convex functions. Only re- 
cently, Wang et al. [36] presented a spectral three-term modification of the 
classical Conjugate Descent scheme [8] with search direction defined as 


= ILgrl|”de—1—(9t du—1) 9% : 
dy =) 9 + mmax(—aT emote 25 
—9k, if k=0, 
where 
= k 2 
Vk = Pk — Il $«—1| > ms 0, 72,73 © (0,1). 


min{n2sf_,Ye—1,3bK}’ 


Now, like the classical PRP [29, 30] and HS [14] methods, the classical LS 
[22] CG scheme is equipped with an in-built mechanism that addresses the 
jamming phenomenon. So, like the others in the group, the scheme is numer- 
ically effective. However, it does not satisfy the sufficient descent condition 
(7). Motivated by this shortcoming of the LS scheme, the nice attributes 
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of three-term methods discussed above and the spectral three-term scheme 
by Wang et al. [36], we propose a spectral three-term adaptation of the LS 
scheme [22] with the following search direction: 


ran FE Gn—1dp—1—F pe de—-19k-1 . 
dy = OEY maa lalla Fs (11) 
—F,, if k=O, 


where 


2 
max Sp ; fs = 
2etol ia Coxe} = SE Tet, Co >0, Cts € (0), (12) 


and 
Yk—-1 = Yr-1 +7Sk-1, Yr-1 = Pe —Fe-1, r>0. 


Note: It can easily be deduced from (11) and (12) that 


max{—dj_ 4 Fi—1,61|%—1lllde—a[]} = Cal Gerla, (13) 


and 


max{¢a||sx—1117,C3xe} © C3Xk _ 
= = (3. 
Xk Xk 
Next, we obtain a bound for the spectral parameter yz. To achieve that, we 
analyze two cases: 


(14) 


First case: If max{¢g|]sx—1||?, 3x“ } = Cal|sx—1||?, then, by monotonicity of 
F, and definition of ¥,_1, we have 


Sh—19k—-1 = 86—-1Yk—-1 + T[I8x-11)? = rllsx—al?. (15) 
So, using (15) we have that 


— max{al|se—ill?, axe} Call-se—all? Z Callsx—-ill? Ge 


Xk 8) tea ileal? or 


Second case: If max{¢3||sx—1||?, 3x4} = (3X, then 


_ max{2||sx—1||?, Csxx} _ 63Xk 
Xk Xk 


=e 


Therefore, setting k = max{*, ¢3}, we see that 0< 4% < kK. 


Algorithm 1. Modified Liu-Storey Method (MLSTM) 


Step 0: Choose a tolerance ¢€ > 0, initial guess xo € ©, 8 > 0, p € (0,1), 
0<¢<2,0>0. Set k = 0 and do = —Fo. 


Step 1: Compute F(a;). If ||F(xz)|| < €, stop, if not, proceed to Step 2. 
Step 2: Find z, = x7, + t,d,, where 
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t, = max{8p”™ :m = 0,1,2,...}, 


for which 
—F (xp + tedp) de > otel|dell, (16) 


Step 3: If z, € ® and ||F(zx)|| < € stop, else determine 
Tep1 = Pe [te — spel (Ze) 5 (17) 


where 
F(z)" (te — 2) 
Il F(zx) II? 
Step 4: Obtain the direction dy41 by (11) and (12). 
Step 5: Set k = k+1. Go to Step 1. 


re (18) 


Lemma 1. Let the sequence {d,} be generated by (11) and (12). Then 


FPdy < —1\|Fi\|?, 7 > 0. (19) 
Proof. First, from (14) we have that y, > ¢3. This implies that —y, < —¢3. 
We now consider two cases: 
1. For k =0, by (11), it is obvious that F@’do = —||Foll?. 
2. For k > 1, from (11) and (12), we have 


(FE Gn—1)dg_1 Fe — (dg_1 Fe) FE Gn-1 


FP dp = —yp\|Fx||? 4 
pe ER Easel at Fa illo alle all 


Setting T = ¢3, we obtain the result in both cases. 


In the following Lemma, we prove that when the solution of (1) is not 
attained, i.e., F(a) #0, then a stepsize t, exists for which (16) is satisfied. 


Lemma 2. Suppose condition (i) in section 2 holds. Then, for every k > 0, 
there exists a positive constant t, such that (16) is satisfied. 


Proof. Assuming that the statement is not true. It implies that a constant 
ko exists for which (16) does not hold for each integer m > 0, namely 


—F (ak, + Bp dk)” dk < 7B0™||dzo||?- 


By employing the continuity of F with the fact that p € (0,1), letting the 
integer m grow to infinity namely, m — oo, we obtain 
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Tf 
—F (ap, )° dk, < 0. (21) 
From (19), we have 


—F(@kq)" Veg > TIF (ko) 


|? > 0. (22) 


Clearly (21) and (22) do not agree. So, a contradiction is obtained, which 
establishes the proof. 


Lemma 3. Let the sequences {x;,} and {z,} be generated by Algorithm 1, 


then 
pT ||Frl| \ 


(E+ adel? -) 


th = min {8 


Proof. By (16), we see that if t, = 6, then (16) is satisfied. Conversely, if 


t, # 8 then th = zs will not satisfy (16), ie., 


t t 
—F(x,_ + 5) di < oF lida. (24) 


By Assumption (di) and (19), we can write 
T||Frll < Ff de 
tk t 
= (F (xp + 5) — F(ax))" dy — F(ae + Bu) di 


tk te (25) 
< L—||dgl|? — o—|[de? 
p p 


t 
= (E+ o)lldnll, 


which ultimately yields the desired inequality and the proof is completed. 


Lemma 4. Let conditions (7) and (7é) in section 2 hold. Then the sequences 
{x,} and {z,} generated by Algorithm 1 are bounded and 


k— oo 


Proof. First, we prove boundedness of the sequences {x,} and {z,}. Let 
& € ® be a solution of (1). Then by (2), we have 


(ap — £)° F(z) = (te — Ze + 2% — £)° F (2x) 
= (ay — 2x)" F (zn) + (2% — #)" F(2e) 
> (an — 2x)" F (zn) + (2 — #)7 F(Z) a 
= (&p- zn)’ F (2x) 


Also from (10), (18), and the fact that 0 < ¢ < 2, we have 
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\|te41 — &||? = || Po(ae — sue (ze)) — &|? 
< |lv~ — sueF (ze) — &Il? 
= ||(a, — 2) — sun F (zx)? 
= |x — &||? — 2cp_F 2K)" (we — &) + 97 yg F (ze) I? (28) 
< || — Ell? — 2c, F(ze)" (en — 2%) + PURE (ee )II? 
‘ (F(zx)? (ax — 2%)? 
= ||z. — 2? — 52-5) 
| F'(ze) II? 
< Lk z||?, 
which consequently yields 
|ze+1 — £|| < ||zx —Z||, Vk > 0. (29) 


And in a recursive manner, (29) implies that ||x, — Z|| < ||ao — Z||. So, the 
sequence {||x, — @||} is decreasing and convergent, which means that {z,} is 
bounded. Also, by assumption (i), (9) and (29) we have 


|F (xe) || = |F (ee) — F(@)I S L]lax — &|| < Ll]2o — aI]. 
Setting L||ao — Z|| = 7, we obtain that 
|F(xx)|| <7. (30) 
Also, from (16) and definition of z,, we get 
F (zx) (ay — ze) = —teF (zu)? dy > ot? ||dg||? = o|lan — ze|l?. (31) 
By employing (2) and the Cauchy-Schwartz inequality, we can write 


F (zx)? (te — 26) = (F (zn) — F(ae))? (te — 2k) + F(ax)* (ae — 2x) 


< ||F(ex)lllen — 2ell. (32) 


By (80),(31) and (32) we can write 
allen — ell? < F(ee)Illlex — zal, 


which leads to = 
Iz — 2el| < — 
o 
Hence, the sequence {z,} is also bounded. Now, the boundedness of {zx}, 
implies that {||z,—||} is bounded, i.e., there exists 72 > 0 such that for any 
LE® 
I|ze — || < m2. (33) 


Similarly, from (9) and (33), we have 
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F(z) ll = Fe) — F@)|| < Lllzn — al] < Dre. 
Hence, setting 73 = L72, we obtain 
IF (zx) || S 73. (34) 
Also, using (16), we have 
oth lidall* < te(F (zx)? du)”. (35) 


By combining (28) and (35), we obtain 


ote lldel||l? < (Ilex — €||? — llwe41 — £II?). (36) 


F(z) I? 
(2 —s) 


Now, by (29) we have that the sequence {||x, — ||} is convergent, and also 
by (34) {F'(z,)} is bounded. Hence, taking limits of both sides of (36) as k 
approaches infinity, we have 


o lim tf|ldell* <0, 
k—- co 
which consequently leads to the desired result, i-e., 


k— oo 


Lemma 5. Let the sequence of search directions {d,} be generated by Al- 
gorithm 1. Then {d,} is bounded, namely, a constant J > 0 exists such 
that 

|dx|| <0, Vk positive. (38) 


Proof. From (11), (13), (30), and the Cauchy-Schwartz inequality, we obtain 


FP o,—1de—-1 — FE de—-19%-1 
I|dx || = || -Y¥e Fe 4 a a 
max{—dj_1Fe—1, ¢1||Gx—1l||dx—1||} 
Fel Ge—a MN N}de—a |] + | Fel Gea Il llde—1l 
< Yell Fall T = 
max{—dy_ 1 Fe—1, 61|Ye—11l||de—1 Il} 
2|| Fr || ||Gx—111 ||dx—1 | 
< yell Fell + = d 
C1||Gx—alllldx—al 
2|| Fx ll 
= Y«e|| Fel] + Te 
1 


< («+ 2) Im 
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<(x+Z)n. (39) 


By setting 0 = (« + 2) m, the proof is established. 


The following theorem establishes global convergence of Algorithm 1. 


Theorem 1. Given that conditions (7) and (iz) hold. Consider the sequences 
{x,} and {z,} generated by Algorithm 1. Then 


lim inf || F},|| = 0. 
k> co 


Proof. For the proof, we assume that the conclusion is not true. Then it 
implies that a constant ¢ > o exists for which 


Fell = ¢, Wk > 0. (40) 
Utilizing (19), (40) and Cauchy Schwartz inequality yields 
I|d;.|| 2 7e, Vk > 0. (41) 


Similarly, by employing the inequalities (23), (39), (40), (41), and for all k& 
sufficiently large, we get 


tulldll > min {9 a} Ide 


> min {ora | > 0. 


Clearly, the second inequality in (42) contradicts (37). Therefore, we conclude 
that lim inf, —. 6 || Fe || = 0. 


4 Numerical experiments and discussions 


Here, we investigate the effectiveness of Algorithm 1 by comparing its per- 
formance with the two methods presented in [16, 18]. The experiments 
with all the three algorithms were conducted using the backtracking line 
search (16). For the other two methods, which we label as MLSCD and 
HCGP for simplicity, we set the parameters as they are used in the re- 
spective papers. For Algorithm 1, we set p = 0.6, 8 = 1, 0 = 107°, 
¢ = 16,r = 1, = 0.5,¢2 = 0.5,¢3 = 0.6. Codes for the algorithms were 
written using Matlab R2014a and run on a PC (2.30GHZ CPU, 4GB RAM). 
The iteration is set to stop for all the methods when the number of iterations 
exceed 1000 or whenever any of the following inequalities is satisfied: 
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||F (xx)|| < 107°, 


IF(zx)I| < 10-8. 
The following test problems were used to test the three methods. 


Problem 4.1. This is a modification of the problem obtained from [34]. The 
mapping F takes the form F(x) = (f1(2), fo(x),.-.; fn(x))7, where 

fi(x) = (e7*)? + 3sinz;-1, i1=2,...,n-1, 

with @= RY". 

Problem 4.2. Exponential Function II obtained from [19]. The mapping F’ 
takes the form F(x) = (f(z), f(x), --; fa(x))7, where 

fila) =e" —1, 4=2,3,...,n, 

fle) 44 (e+e — 1); 

with 6=R'. 

Problem 4.3. Non-smooth Function obtained from[20]. The mapping F’ 
takes the form F(x) = (fi (2), fo(2), ..., fr(x))", where 


fi(x) = 2a; —sin|x;|, t=1,2,...,n, 
with ® = rer": don <n, 2; >0, fat aennh, 
i=1 
Problem 4.4. Strictly Convex Function obtained from [34]. The mapping 
F takes the form F(x) = (f1(2), f(x), .-.; fr(x))7, where 
fi(x) =e" —1, 1=1,2,...,n, 
with 6=R'". 
Problem 4.5. Tridiagonal Exponential Function obtained from [23]. The 
mapping F takes the form F(x) = (f1(2), fo(x),.-.; fa(x))7, where 


fi(z) = 21 — elcos*n"), 
(cos *#=14eiF i+ ) : 
fi(x) = 2; —e at , 2=2,3,...,n—-1, 
fn (2) (constr) 
nv) = L,—e hs 


with 6=R‘. 
Problem 4.6. Non-smooth Function obtained from [46]. The mapping F’ 
takes the form F(x) = (fi(2), fo(a),..., fr(x))", where 
fi(x) =a; —sin|a;—-1], *+=1,2,...,n, 
with ® = rer": don <n, 2,>0, fat aennh, 
i=1 
Problem 4.7 The problem is obtained from [19]. The mapping F’ takes the 
form F(0) = (fi(2), fo(2),-- fa(2))", where 
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f(x) =e" — 1, 
fi(x) =e" +a;4-1—-1, 4=2,...,n—-—1, 
with 6=R'". 


Problem 4.8 Modified version of the non-smooth Function in Problem 4.6. 
The mapping F takes the form F(a) = (fi(2), fo(«),.-.; fn(w))7, where 


fi(x) =a; —2sin|z;-1], t=1,2,...,n, 
wih a= [re R":Son en x; > 0, fat aennh, 
i=1 


For each of the above test functions, 24 numerical experiments were per- 
formed with variables 1000, 10,000 50, 000, and the following starting points: 


BiSises8) 6 = Adin)"; 
CT Tiss d) sig = 18, Sreeg BI” 9s 


3 
oe 
,ty = 


= (5,9, 040) «06S (0, Gp, 0)". 


a 


Figure 1: Performance profile with respect to number of iterations 


Furthermore, we adopt Dolan and Moré [6] performance profile in order 
to present a graphical view of the performance of each of the three schemes 
considered in the experiments. In line with this, three figures were plotted 
with respect to three performance metrics, namely, number of iterations, 
function evaluations and processing time. For each figure, the vertical — axis 
corresponds to the percentage of the problems solved by any one of the al- 
gorithms with the least value of any of the metric under consideration; the 
right side, represents the percentage of problems solved successfully by each 
algorithm. Also, the topmost curve in each figure corresponds to the algo- 
rithm that solved the most problems in the experiments. It can be observed 
from Fig. 1, that the MLSTM algorithm solved 78% of problems with least 
number of iterations compared to the MLSCD and HCGP algorithms that 
recorded 62% and 54% respectively. We have to note here, that this percent- 
age values as shown in the figure, represent sums of the percentages recorded 
by each algorithm with least number of iterations and the ties recorded for 
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MLSTM| 


L L L L L L L 
0 0.5 1 1.5 2 25 3 3.5 4 


a 


Figure 3: Performance profile with respect to CPU time 


the same metric. From Fig. 2 it is observed that the MLSTM algorithm 
solved 65% of the problems with least function evaluations compared to the 
HCGP and MLSCD algorithms that recorded 52% and 4% respectively. 
As in the earlier case, here too the percentage values are sums of the val- 
ues recorded by each algorithm with least function values and the ties it 
recorded with any one or two other algorithms. Based on least processing 
time metric, Fig. 3 indicated that the HCGP algorithm has an edge over 
the MLSTM and MLSCD algorithms as it solved 45% of the problems with 
minimum processing time, while the other two algorithms recorded 40% and 
18% respectively. Moreover, it is observed that the topmost curve in all the 
three figures corresponds to the MLZSTM algorithm. Hence, considering the 
graphical representations in Figs 1, 2, and 3, and the above analysis, it can 
be concluded that the MLSTM algorithm is more effective for solving the 
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problem represented in (1) than the MZSCD and HCGP algorithms. 


5 Application of the proposed scheme 


Obtaining sparse solutions to ill-conditioned linear systems of equations is 
the interest in most signal and image processing problems. Typically, this 
involves minimizing the following ¢; — 02 norm problem 


_ ol 
min 5||Ha — wl|3 + d|lelli, (43) 


where 6 is a nonnegative parameter, z € R”,w € R* is an observed value, 
H € R**"(k <n) denotes a linear mapping, while ||2||; and ||a||2 repre- 
sents the @; and 2 norms respectively. Clearly, (43) represents a convex 
unconstrained optimization problem. 


In order to solve (43), Figueiredo et al. [7] reformulated it as a convex 
quadratic problem, where each vector z € R” is split into two parts and 
written as 


cr=a-—b, a>O0, b>0, ade R”. (44) 
with a; = (@;)4,0; = (—a,)+4, Vi = 1,2,...,n and (.)+ = max{0,x}. Applying 
the above representation to (43), we obtain 


1 
min gH (a— 4) — w||2 + deZa + de7b, (45) 


where e, = (1,1,...,1)7 € R”. Going by Figueiredo et al. [7], the problem 
in (45) is reformulated as 


1 
min 52 Ae + Dz, z>0, (46) 


which is a quadratic program problem with 


_ (4% _ -y _ TP _ ( HH —-H'H 
-=(§), D=dem+ (FY), y=Hu, Ge HTH 7 
(47) 
In [45], the quadratic program problem in (46) was reformulated and shown 


to be equivalent to 
F(z) = min{z, Az + D} =0, (48) 


where F represents a vector-valued mapping. Also, since F' is monotone and 
Lipschitz continuous (see [28, 45]), the MLSTM scheme can conveniently be 
applied to solve it. 
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5.1 Image restoration experiment 


Here, we conduct some experiments with the MLISTM scheme to further 
demonstrate its effectiveness and application in image reconstruction. For 
the experiments, four images are employed, which includes Barbera, Lena, 
Einstein, and Cameraman. As in the previous experiments, all codes are 
generated on MATLAB R2014a with the same configuration and parameter 
values set as applied in the earlier experiments. Also, we test performance 
of the MLSTM method with the CGDESCENT [46] solver, which is used 
in image restoration problems. The same values of the parameters used by 
the author were also applied in the experiments. The performance of both 
schemes are compared in terms of final objective function value (Obj), mean 
square error (MSE), signal to noise ratio (SNR), which is given by 


|=" 
SNR = 20 x logy a ; 


and the structural similarity index (SSIM), which computes the similarity 
between original image and the restored one in each of the experiments con- 
ducted. Results of the experiments conducted are presented in table 1, while 
Fig. 4 displays the original, blurred, and reconstructed images obtained by 
the MLSTM and CGDESCENT schemes. Fig. 4 reveals that the quality 
of reconstructed images by the MLZSTM method for all the images consid- 
ered, is somewhat better than that of CGDESCENT scheme. Also, table 1 
showed that the MLSTM algorithm performed much better regarding Obj, 
MSE, SNR and SSIM metrics than the CGDESCENT scheme. However, 
the CG@DESCENT scheme is much faster as it recorded less processing time 
than the MLSTM algorithm. Hence, going by these results, it can be con- 
cluded that the MZSTM algorithm is promising for reconstruction of the 
images considered. 


Table 1: Image restoration results for MLSTM and CGDESCENT 


IMAGE SIZE MLSTM CGDESCENT 
Obj MSE SNR PT SSIM Obj MSE SNR PT SSIM 

BARBERA — 256 x 256 1.523 x 10° 1.5267 x 10? 20.85 10.75 0.80 1.585 x 10° 2.0627 x 10? 19.55 1.45 0.75 
LENA 256 x 256 1.448 x 10° 6.4566 x 10! 24.37 17.81 0.90 1.513 x 10° 9.0025 x 10! 22.93 1.81 0.87 
EINSTEIN 256 x 256 1.012 x 10° 7.0780 x 10! 21.39 10.33 0.86 1.045 x 10° 8.7884 x 10! 20.45 3.59 0.83 
CAMERAMAN 256 x 256 1.430 x 10° 1.4128 x 10? 21.05 9.59 0.85 1.47x 10° 1.78 x10? 20.05 3.14 0.83 


6 Conclusion 


An efficient modified Liu-Storey scheme (MLSTM) for solving constrained 
system of monotone nonlinear equations was presented in this article. The 
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CGDESCENT 


CGDESCENT 


Figure 4: Original and blurred images (First and second columns from the left). Restored 
images by MLSTM and CGDESCENT methods (last two columns ). 


scheme is ideal for large dimension problems as well as non-smooth functions 
because it avoids computing derivatives and requires less memory to imple- 
ment. Apart from inheriting numerical efficiency of the classical LS scheme, 
the new method satisfies the important condition for global convergence. The 
scheme’s global convergence was established by employing basic assumptions. 
Also, numerical experiments conducted with some test problems indicate that 
the proposed scheme is promising as it is competitive and more efficient com- 
pared to the MLSCD and HCGP methods in [16, 18]. Furthermore, an 
interesting novelty of the scheme is its application to solving the regularized 
£, norm problem in compressed sensing. By conducting some experiments 
to recover blurry images, the scheme proved to be effective as it competes 
with and produces better results than the popular CGDESCENT scheme in 
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[46]. As a further research, we intend to explore application of the MLSTM 
scheme and its modified version to other area of interest. 
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